Benchmarking Ontology Supplement

In the field of information retrieval (IR), the goal is to identify documents from a large collection that are most relevant to a user’s query. If the subset of relevant documents is known, we can evaluate the quality of an information retrieval method using the metrics of precision, recall, accuracy, fallout and the F -measure (harmonic mean of precision and recall). To define these metrics we need to determine the true positives (tp), false positives (fp), true negatives (tn), and false negatives (fn) achieved by the retrieval method. These are defined by the cross-tabulation between relevance and retrieval: true positives comprise documents that are both relevant to the query and retrieved by the method; false positives are documents retrieved but irrelevant; false negatives are relevant but not retrieved; and true negatives include all irrelevant documents not retrieved by the method.